1. Technical Field
This invention relates generally to computer software applications, and more particularly to instrumenting object oriented programmed applications. The invention more specifically relates to instrumenting object oriented program applications to provide application-related information consisting of indications of end user interactions with the applications and how the end user makes use of the applications.
2. Prior Art
It is common experience that most users of computing applications encounter various sorts of problems and inconsistencies while using the applications. Designing computer software that will minimize user problems requires knowledge as to how the user is making use of the application and the tasks that are trying to be accomplished by use of the applications. The design process typically includes iterative design and user testing of those designs in order to identify where users have problems so that design flaws can be accurately addressed and fixed. A thorough knowledge of problems that users encounter in existing products is also critical if design mistakes are to be avoided for future versions of those applications.
Understanding what is important to users of a particular software application product has always been a challenge for software application developers. Typically, enhancements to a product release are based on feedback from surveys, forums or from user design centre sessions in which participants are given a set of tasks to accomplish while using the application. Some problems inherent in these methods, however, include the inability of users to articulate their needs, the artificiality of surveys produces feedback which is not meaningful, and users of forums are typically technical fans of the product so that the results may be of little practical use to the product developers.
Most would agree that the best way to get feedback is by observing what a user does throughout actual use sessions with the product, that is by watching the user as the user works through the application in the setting for which it was intended. The problem is to provide a meaningful solution for gathering real user feedback in real time on a regular basis, and providing this data back to the program developer where it can be analyzed and used in enhancing and improving the application. One effective way of gathering such information about user interaction with the application is through code instrumentation. The term "instrumentation" is understood to mean the incorporation of statements or routines in an application to cause outputs to be provided when the application is running on a computer system thereby illustrating the interaction of the end user with the application.
Currently, it is known that instrumenting an application, including an object oriented programmed application, involves going into the application and adding appropriate program code at every point in the application where useful user data can be gathered, and then recompiling the application. It has been found that this "compile-time" method of instrumentation is not really effective for a number of reasons. It takes many points of instrumentation to get sufficient and meaningful overall coverage of user interaction with the application. At the particular code level, it is difficult to be judicious in selecting code points for instrumentation. A large number of instrumentation points can negatively affect the application's performance. The amount of data produced depends entirely on where the user is spending time in working with the application. In some instances, the data can be large and therefore cumbersome to analyze, and at other times, it may be sparse and therefore of very little practical use.
Any fine tuning of the instrumentation to adjust for the aforementioned negative results involves farther coding and re-compiling of the application. In addition, a revised application may have to be delivered to the end user. Probably the most significant obstacle to the effectiveness of this known method is that instrumenting application code requires a programmer who has expert knowledge of that code. Such a person typically is rarely involved in user feedback activities. There is no previously known solution to readily overcome the aforementioned inherent problems with "compile-time" instrumentation of a program application. The "run-time" instrumentation to be subsequently described herein to obtain feedback from users of applications has not been previously suggested.
The most general example of "compile-time" instrumentation is simply where source code statements or routines are added to a source code version of a software application, and then the revised application is compiled.
It is a known software development technique that tracing information resulting from the use of compile-time instrumentation is sometimes used by developers to "debug" programs to provide state information of running applications. However, this necessitates the recompiling of the application with each addition of instrumentation code.
As an example of instrumentation of a program, reference is made to U.S. Pat. No. 5,313,616 which issued May 17, 1994 to 88Open Consortium Ltd., entitled "Method for Analyzing Calls of Application Program by Inserting Monitoring Routines into the Executable Version and Redirecting Calls to the Monitoring Routines". This reference relates to testing applications resulting from process-based programming technology to determine the application's conformance with some standard or set of operating system rules. The method entails that all possible instrumentation points are located in the application and monitor code is inserted at each point in order to be able to carry out a dynamic analysis of the application. The monitor code is added to the binary version of the application and thus the application is changed after it has been compiled and linked.
Other methods and systems which make use of instrumentation techniques are known for evaluating and testing the performance of computer programs. These are generally referred to as software evaluation tools. One such representative system is described in U.S. Pat. No. 5,465,258, entitled "Binary Image Performance Evaluation Tool" which issued to Integrity Systems, Inc. on Nov. 7, 1995. This patent relates to evaluating the performance of a particular computer program on a particular computer by processing an executable version of the program and an architecture description file of the computer resulting in an instrumented image of the program code. Execution of the instrumented image results performance information being collected for subsequent analysis and assessment.
Another representative evaluation system is described in U.S. Pat. No. 5,581,696 entitled "Method Using a Computer for Automatically Debugging" which issued Dec. 3, 1996 to Parasoft Corp. This specification provides for the automatic instrumenting of a computer program with debug routines for dynamic debugging. By processing a source program file and a programming language definition file an executable program file is obtained which contains the instrumented program. The teachings of this reference may be considered to be compile-time instrumentation.